HAFNet: Hierarchical Attentive Fusion Network for Multispectral Pedestrian Detection
نویسندگان
چکیده
Multispectral pedestrian detection via visible and thermal image pairs has received widespread attention in recent years. It provides a promising multi-modality solution to address the challenges of low-light environments occlusion situations. Most existing methods directly blend results two modalities or combine features linear interpolation. However, such fusion strategies tend extract coarser corresponding positions different modalities, which may lead degraded performance. To mitigate this, this paper proposes novel adaptive cross-modality framework, named Hierarchical Attentive Fusion Network (HAFNet), fully exploits multispectral knowledge inspire decision-making process. Concretely, we introduce Content-dependent (HCAF) module top-level as guide pixel-wise blending enhance quality feature representation plug-in alignment (MFA) block fine-tune modalities. Experiments on challenging KAIST CVC-14 datasets demonstrate superior performance our method with satisfactory speed.
منابع مشابه
Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks
Robust vision-based pedestrian detection is a crucial feature of future autonomous systems. Thermal cameras provide an additional input channel that helps solving this task and deep convolutional networks are the currently leading approach for many pattern recognition problems, including object detection. In this paper, we explore the potential of deep models for multispectral pedestrian detect...
متن کاملMultispectral Deep Neural Networks for Pedestrian Detection
Multispectral pedestrian detection is essential for around-the-clock applications, e.g., surveillance and autonomous driving. We deeply analyze Faster R-CNN for multispectral pedestrian detection task and then model it into a convolutional network (ConvNet) fusion problem. Further, we discover that ConvNet-based pedestrian detectors trained by color or thermal images separately provide compleme...
متن کاملFusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection
Multispectral pedestrian detection has received extensive attention in recent years as a promising solution to facilitate robust human target detection for around-the-clock applications (e.g. security surveillance and autonomous driving). In this paper, we demonstrate illumination information encoded in multispectral images can be utilized to significantly boost performance of pedestrian detect...
متن کاملIllumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection
Multispectral images of color-thermal pairs have shown more effective than a single color channel for pedestrian detection, especially under challenging illumination conditions. However, there is still a lack of studies on how to fuse the two modalities effectively. In this paper, we deeply compare six different convolutional network fusion architectures and analyse their adaptations, enabling ...
متن کاملSelf-Attentive Feature-level Fusion for Multimodal Emotion Detection
Multimodal emotion recognition is the task of detecting emotions present in user-generated multimedia content. Such resources contain complementary information in multiple modalities. A stiff challenge often faced is the complexity associated with feature-level fusion of these heterogeneous modes. In this paper, we propose a new feature-level fusion method based on self-attention mechanism. We ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Remote Sensing
سال: 2023
ISSN: ['2315-4632', '2315-4675']
DOI: https://doi.org/10.3390/rs15082041